Dr Andrew Bradley

Interpretation and understanding of video presents a challenging computer vision task in numerous fields - e.g. autonomous driving and sports analytics. Existing approaches to interpreting the actions taking place within a video clip are based upon Temporal Action Localisation (TAL), which typically identifies short-term actions. The emerging field of Complex Activity Detection (CompAD) extends this analysis to long-term activities, with a deeper understanding obtained by modelling the internal structure of a complex activity taking place within the video.We address the CompAD problem using a hybrid graph neural network which combines attention applied to a graph encoding the local (short-term) dynamic scene with a temporal graph modelling the overall long-duration activity. Our approach is as follows: i) Firstly, we propose a novel feature extraction technique which, for each video snippet, generates spatiotemporal ‘tubes’ for the active elements (‘agents’) in the (local) scene by detecting individual objects, tracking them and then extracting 3D features from all the agent tubes as well as the overall scene. ii) Next, we construct a local scene graph where each node (representing either an agent tube or the scene) is connected to all other nodes. Attention is then applied to this graph to obtain an overall representation of the local dynamic scene. iii) Finally, all local scene graph representations are interconnected via a temporal graph, to estimate the complex activity class together with its start and end time.The proposed framework outperforms all previous state-of-the-art methods on all three datasets including ActivityNet-1.3, Thumos-14, and ROAD.

The emerging field of action prediction - the task of forecasting action in a video sequence - plays a vital role in various computer vision applications such as autonomous driving, activity analysis and human-computer interaction. Despite significant advancements, accurately predicting future actions remains a challenging problem due to high dimensionality, complex dynamics and uncertainties inherent in video data. Traditional supervised approaches require large amounts of labelled data, which is expensive and time-consuming to obtain. This paper introduces a novel self-supervised video strategy for enhancing action prediction inspired by DINO (self-distillation with no labels). The approach, named Temporal-DINO, employs two models; a ‘student’ processing past frames; and a ‘teacher’ processing both past and future frames, enabling a broader temporal context. During training, the teacher guides the student to learn future context by only observing past frames. The strategy is evaluated on ROAD dataset for the action prediction downstream task using 3D-ResNet, Transformer, and LSTM architectures. The experimental results showcase significant improvements in prediction performance across these architectures, with our method achieving an average enhancement of 9.9% Precision Points (PP), which highlights its effectiveness in enhancing the backbones’ capabilities of capturing long-term dependencies. Furthermore, our approach demonstrates efficiency in terms of the pretraining dataset size and the number of epochs required. This method overcomes limitations present in other approaches, including the consideration of various backbone architectures, addressing multiple prediction horizons, reducing reliance on hand-crafted augmentations, and streamlining the pretraining process into a single stage. These findings highlight the potential of our approach in diverse video-based tasks such as activity recognition, motion planning, and scene understanding. Code can be found at https://github.com/IzzeddinTeeti/ssl_pred.

This paper proposes a scenario-based functional testing approach for enhancing the performance of machine learning (ML) applica- tions. The proposed method is an iterative process that starts with testing the ML model on various scenarios to identify areas of weakness. It follows by a further testing on the suspected weak scenarios and statistically evaluate the model’s performance on the scenarios to confirm the diagnosis. Once the diagnosis of weak scenarios is confirmed by test results, the treatment of the model is performed by retraining the model using a transfer learning technique with the original model as the base and applying a set of training data specifically targeting the treated scenarios plus a subset of training data selected at random from the original train dataset to prevent the so-call catastrophic forgetting effect. Finally, after the treatment, the model is assessed and evaluated again by testing on the treated scenarios as well as other scenarios to check if the treatment is effective and no side-effect caused. The paper reports a case study with a real ML deep neural network (DNN) model, which is the perception system of an autonomous racing car. It is demonstrated that the method is effective in the sense that DNN model’s performance can be improved. It provides an efficient method of enhancing ML model’s performance with much less human and compute resource than retrain from scratch.

Connected and Autonomous Vehicles (CAVs) rely on Vehicular Adhoc Networks with wireless communication between vehicles and roadside infrastructure to support safe operation. However, cybersecurity attacks pose a threat to VANETs and the safe operation of CAVs. This study proposes the use of simulation for modelling typical communication scenarios which may be subject to malicious attacks. The Eclipse MOSAIC simulation framework is used to model two typical road scenarios, including messaging between the vehicles and infrastructure - and both replay and bogus information cybersecurity attacks are introduced. The model demonstrates the impact of these attacks, and provides an open dataset to inform the development of machine learning algorithms to provide anomaly detection and mitigation solutions for enhancing secure communications and safe deployment of CAVs on the road.

The detection of road agents, such as vehicles and pedestrians are central in autonomous driving. Self-Supervised Learning (SSL) has been proven to be an effective technique for learning discrimi-native feature representations for image classification , alleviating the need for labels, a remarkable advancement considering how time-consuming and expensive labelling can be in autonomous driving. In this paper, we investigate the effectiveness of contrastive SSL techniques such as BYOL and MoCo on the object (agent) detection task using the ROad event Awareness Dataset (ROAD). Our experiments show that using self-supervised learning we can achieve a 3.96% improvement on the AP@50 metric for agent detection compared to supervised pretraining. Extensive comparisons and evaluations of current state-of-the-art SSL methods (namely MOCO, BYOL, SCRL) are conducted and reported for the object detection task.

This survey targets intention and trajectory prediction in Autonomous Vehicles (AV), as AV companies compete to create dedicated prediction pipelines to avoid collisions. The survey starts with a formal definition of the prediction problem and highlights its challenges, to then critically compare the models proposed in the last 2-3 years in terms of how they overcome these challenges. Further, it lists the latest methodological and technical trends in the field and comments on the efficacy of different machine learning blocks in modelling various aspects of the prediction problem. It also summarises the popular datasets and metrics used to evaluate prediction models, before concluding with the possible research gaps and future directions.

In an autonomous driving system, perception - identification of features and objects from the environment
- is crucial. Autonomous racing, in particular, features high speeds and small margins that demand rapid and accurate perception systems. During the race, the weather can change abruptly, causing significant degradation in perception, resulting in ineffective manoeuvres. In order to improve detection in adverse weather, deep-learning-based models typically require extensive datasets captured in such conditions - the
collection of which is a protracted and costly process. However, recent developments in Cycle-GAN architectures allow the synthesis of highly realistic scenes in multiple weather conditions. To this end, we introduce an approach of using synthesised adverse condition datasets in autonomous racing (generated using CycleGAN) to improve the performance of four out of five state-of-the-art detectors by an average of 42.7 and 4.4 mean average precision (mAP) percentage points in the presence of night-time conditions and droplets, respectively. Furthermore, we present a comparative analysis of five object detectors - identifying
the optimal pairing of detector and training data for use during autonomous racing in challenging conditions.

Autonomous vehicles make use of sensors to perceive the world around them, with heavy reliance on visionbased sensors such as RGB cameras. Unfortunately, since these sensors are affected by adverse weather, perception pipelines require extensive training on visual data under harsh conditions in order to improve the robustness of downstream tasks - data that is difficult and expensive to acquire. Based on GAN and CycleGAN architectures, we propose an overall (modular) architecture for constructing datasets, which allows one to add, swap out and combine components in order to generate images with diverse weather conditions. Starting from a single dataset with ground-truth, we generate 7 versions of the same data in
diverse weather, and propose an extension to augment the generated conditions, thus resulting in a total of 14 adverse weather conditions, requiring a single ground truth. We test the quality of the generated conditions both in terms of perceptual quality and suitability for training downstream tasks, using real world, out-of-distribution adverse weather extracted from various datasets. We show improvements in both object detection and instance segmentation across all conditions, in many cases exceeding 10 percentage points
increase in AP, and provide the materials and instructions needed to re-construct the multi-weather dataset, based upon the original Cityscapes dataset.

The rising popularity of autonomous vehicles has led to the development of driverless racing cars, where the
competitive nature of motorsport has the potential to drive innovations in autonomous vehicle technology. The challenge of racing requires the sensors, object detection and vehicle control systems to work together at the highest possible speed and computational efficiency. This paper describes an autonomous driving system for a self-driving racing vehicle application using a modest sensor suite coupled with accessible processing hardware, with an object detection system capable of a frame rate of 25fps, and a mean average precision of 92%. A modelling tool is developed in open-source software for real-time dynamic simulation of the autonomous vehicle and associated sensors, which is fully interchangeable with the real vehicle. The simulator provides performance metrics, which enables accelerated and enhanced quantitative analysis, tuning and optimisation of the autonomous control system algorithms. A design study demonstrates the ability of the simulation to assist in control system parameter tuning - resulting in a 12% reduction in lap time, and an average velocity of 25 km/h - indicating the value of using simulation for the optimisation of multiple parameters in the autonomous control system.

Climate change is the defining challenge now facing our planet. Limiting global warming to 1.5 degrees, as advocated by the Intergovernmental Panel on Climate Change, requires rapid, far-reaching, and unprecedented changes in how governments, industries, and societies function by 2030. Computer Science plays an important role in these efforts, both in providing tools for greater understanding of climate science and in reducing the environmental costs of computing. It is vital for Computer Science students to understand how their chosen field can both exacerbate and mitigate the problem of climate change.

We have reviewed the existing literature, interviewed leading experts, and held conversations at the ITiCSE 2019 conference, to identify how universities, departments, and CS educators can most effectively address climate change within Computer Science education. We find that the level of engagement with the issue is still low, and we discuss obstacles at the level of institutional, program and departmental support as well as faculty and student attitudes. We also report on successful efforts to date, and we identify responses, strategies, seed ideas, and resources to assist educators as they prepare their students for a world shaped by climate change.

A multibody model of an electric go-kart was developed in Msc-Adams Car software to simulate the vehicle’s dynamic performance. In contrast to an ICE kart, its electric counterpart bares an extra weight load accounted for the batteries and other powertrain components. The model is inspired on a prototype vehicle developed at Universidad de los Andes. The prototype was built on top of an ICE frame where a PMAC motor, controller, battery pack and the subsequent powertrain components were installed. A petrol-based Go-kart weight distribution was defined as baseline and several variants of the electric adaptation with different weight distributions were constructed. The main objective of the model is to evaluate different configurations and identify the ones that can give performance advantages. Step steer simulations ran at 40 km/h (64 mph) were analyzed to assess the dynamic performance of the vehicle for different configuration of the battery bank placement.

For most iterations of powertrain location, considerable differences in dynamic response were obtained and the handling balance was identified as Understeer contrary to a priori thoughs. Understeer gradient, weight distribution for both axles, trajectory among other results of interest were observed in the simulations. The model allowed to showcase the effect of redistribution of weight on the dynamic behavior in this specific application. Among the main consequences lies the fact that battery distribution can affect the lifting of the internal rear tire and the detriment in turning effectiveness.

As autonomous vehicles and autonomous racing rise in popularity, so does the need for faster and more accurate detectors. While our naked eyes are able to extract contextual information almost instantly, even from far away, image resolution and computational resources limitations make detecting smaller objects (that is, objects that occupy a small pixel area in the input image) a truly challenging task for machines and a wide open research field. This study explores ways in which the popular YOLOv5 object detector can be modified to improve its performance in detecting smaller objects, with a particular focus on its application to autonomous racing. To achieve this, we investigate how replacing certain structural elements of the
model (as well as their connections and other parameters) can affect performance and inference time. In doing so, we propose a series of models at different scales, which we name ‘YOLO-Z’, and which display an improvement of up to 6.9% in mAP when detecting smaller objects at 50% IOU, at a cost of just a 3ms increase in inference time compared to the original YOLOv5. Our objective is not only to inform future research on the potential of adjusting a popular detector such as YOLOv5 to address specific tasks, but also to provide insights on how specific changes can impact small object detection. Such findings, applied to the wider context of autonomous vehicles, could increase the amount of contextual information available to such systems.